Switchable LED-based laparoscopic multispectral system for rapid high-resolution perfusion imaging

SIGNIFICANCE: Multispectral imaging (MSI) is an approach for real-time, quantitative, and non-invasive tissue perfusion measurements. Current laparoscopic systems based on mosaic sensors or filter wheels lack high spatial resolution or acceptable frame rates.AIM: To develop a laparoscopic system for MSI-based color video and tissue perfusion imaging during gastrointestinal surgery without compromising spatial or temporal resolution.APPROACH: The system was built with 14 switchable light-emitting diodes in the visible and near-infrared spectral range, a 4K image sensor, and a 10 mm laparoscope. Illumination patterns were created for tissue oxygenation and hemoglobin content monitoring. The system was calibrated to a clinically approved laparoscopic hyperspectral system using linear regression models and evaluated in an occlusion study with 36 volunteers.RESULTS: The root mean squared errors between the MSI and reference system were 0.073 for hemoglobin content, 0.039 for oxygenation in deeper tissue layers, and 0.093 for superficial oxygenation. The spatial resolution at a working distance of 45 mm was 156  μm. The effective frame rate was 20 fps.CONCLUSIONS: High-resolution perfusion monitoring was successfully achieved. Hardware optimizations will increase the frame rate. Parameter optimizations through alternative illumination patterns, regression, or assumed tissue models are planned. Intraoperative measurements must confirm the suitability during surgery.</p

Automatic Recognition of Colon and Esophagogastric Cancer with Machine Learning and Hyperspectral Imaging

There are approximately 1.8 million diagnoses of colorectal cancer, 1 million diagnoses of stomach cancer, and 0.6 million diagnoses of esophageal cancer each year globally. An automatic computer-assisted diagnostic (CAD) tool to rapidly detect colorectal and esophagogastric cancer tissue in optical images would be hugely valuable to a surgeon during an intervention. Based on a colon dataset with 12 patients and an esophagogastric dataset of 10 patients, several state-of-the-art machine learning methods have been trained to detect cancer tissue using hyperspectral imaging (HSI), including Support Vector Machines (SVM) with radial basis function kernels, Multi-Layer Perceptrons (MLP) and 3D Convolutional Neural Networks (3DCNN). A leave-one-patient-out cross-validation (LOPOCV) with and without combining these sets was performed. The ROC-AUC score of the 3DCNN was slightly higher than the MLP and SVM with a difference of 0.04 AUC. The best performance was achieved with the 3DCNN for colon cancer and esophagogastric cancer detection with a high ROC-AUC of 0.93. The 3DCNN also achieved the best DICE scores of 0.49 and 0.41 on the colon and esophagogastric datasets, respectively. These scores were significantly improved using a patient-specific decision threshold to 0.58 and 0.51, respectively. This indicates that, in practical use, an HSI-based CAD system using an interactive decision threshold is likely to be valuable. Experiments were also performed to measure the benefits of combining the colorectal and esophagogastric datasets (22 patients), and this yielded significantly better results with the MLP and SVM models

Novel Intraoperative Imaging of Gastric Tube Perfusion during Oncologic Esophagectomy—A Pilot Study Comparing Hyperspectral Imaging (HSI) and Fluorescence Imaging (FI) with Indocyanine Green (ICG)

Background: Novel intraoperative imaging techniques, namely, hyperspectral (HSI) and fluorescence imaging (FI), are promising with respect to reducing severe postoperative complications, thus increasing patient safety. Both tools have already been used to evaluate perfusion of the gastric conduit after esophagectomy and before anastomosis. To our knowledge, this is the first study evaluating both modalities simultaneously during esophagectomy. Methods: In our pilot study, 13 patients, who underwent Ivor Lewis esophagectomy and gastric conduit reconstruction, were analyzed prospectively. HSI and FI were recorded before establishing the anastomosis in order to determine its optimum position. Results: No anastomotic leak occurred during this pilot study. In five patients, the imaging methods resulted in a more peripheral adaptation of the anastomosis. There were no significant differences between the two imaging tools, and no adverse events due to the imaging methods or indocyanine green (ICG) injection occurred. Conclusions: Simultaneous intraoperative application of both modalities was feasible and not time consuming. They are complementary with regard to the ideal anastomotic position and may contribute to better surgical outcomes. The impact of their simultaneous application will be proven in consecutive prospective trials with a large patient cohort

Spectral similarity measures for in vivo human tissue discrimination based on hyperspectral imaging

Problem: Similarity measures are widely used as an approved method for spectral discrimination or identification with their applications in different areas of scientific research. Even though a range of works have been presented, only a few showed slightly promising results for human tissue, and these were mostly focused on pathological and non-pathological tissue classification. Methods: In this work, several spectral similarity measures on hyperspectral (HS) images of in vivo human tissue were evaluated for tissue discrimination purposes. Moreover, we introduced two new hybrid spectral measures, called SID-JM-TAN(SAM) and SID-JM-TAN(SCA). We analyzed spectral signatures obtained from 13 different human tissue types and two different materials (gauze, instruments), collected from HS images of 100 patients during surgeries. Results: The quantitative results showed the reliable performance of the different similarity measures and the proposed hybrid measures for tissue discrimination purposes. The latter produced higher discrimination values, up to 6.7 times more than the classical spectral similarity measures. Moreover, an application of the similarity measures was presented to support the annotations of the HS images. We showed that the automatic checking of tissue-annotated thyroid and colon tissues was successful in 73% and 60% of the total spectra, respectively. The hybrid measures showed the highest performance. Furthermore, the automatic labeling of wrongly annotated tissues was similar for all measures, with an accuracy of up to 90%. Conclusion: In future work, the proposed spectral similarity measures will be integrated with tools to support physicians in annotations and tissue labeling of HS images

A generic concept for the development of model-guided clinical decision support systems

Disease development and progression are very complex processes which make clinical decision making non-trivial. On the one hand, examination results that are stored in multiple formats and data types in clinical information systems need to be considered. Beyond, biological or molecular-biological processes can influence clinical decision making. So far, biological knowledge and patient data is separated from each other. This complicates inclusion of all relevant knowledge and information into the decision making. In this paper, we describe a concept of model-based decision support that links knowledge about biological processes, treatment decisions and clinical data. It consists of three models: 1) a biological model, 2) a decision model encompassing medical knowledge about the treatment workflow and decision parameters, and 3) a patient data model generated from clinical data. Requirements and future steps for realizing the concept will be presented and it will be shown how the concept can support the clinical decision making

MODELISATION TOPOLOGIQUE 3D DE L'ARBRE CORONAIRE POUR L'ETIQUETAGE AUTOMATIQUE EN ANGIOGRAPHIE PAR RAYONS X (DOCTORAT (ELECTRONIQUE, ELECTROTECHNIQUE, AUTOMATIQUE))

LYON1-BU Santé (693882101) / SudocPARIS-BIUM (751062103) / SudocPARIS-BIUP (751062107) / SudocSudocFranceF

Automatic depth scanning system for 3D infrared thermography

Infrared thermography can be used as a pre-, intra- and post-operative imaging technique during medical treatment of patients. Modern infrared thermal cameras are capable of acquiring images with a high sensitivity of 10 mK and beyond. They provide a planar image of an examined 3D object in which this high sensitivity is only reached within a plane perpendicular to the camera axis and defined by the focus of the lens. Out of focus planes are blurred and temperature values are inaccurate. A new 3D infrared thermography system is built by combining a thermal camera with a depth camera. Multiple images at varying focal planes are acquired with the infrared camera using a motorized system. The sharp regions of individual images are projected onto the 3D object’s surface obtained by the depth camera. The system evaluation showed that deviation between measured temperature values and a ground truth is reduced with our system

Development of a tool-kit for the detection of healthy and injured cardiac tissue based on MR imaging

Planning of interventions to treat cardiac arrhythmia requires a 3D patient specific model of the heart. Currently available commercial or free software dedicated to this task have important limitations for routinely use. Automatic algorithms are not robust enough while manual methods are time-consuming. Therefore, the project attempts to develop an optimal software tool. The heart model is generated from preoperative MR data-sets acquired with contrast agent and allows visualisation of damaged cardiac tissue. A requirement in the development of the software tool was the use of semi-automatic functions to be more robust. Once the patient image dataset has been loaded, the user selects a region of interest. Thresholding functions allow selecting the areas of high intensities which correspond to anatomical structures filled with contrast agent, namely cardiac cavities and blood vessels. Thereafter, the target-structure, for example the left ventricle, is coarsely selected by interactively outlining the gross shape. An active contour function adjusts automatically the initial contour to the image content. The result can still be manually improved using fast interaction tools. Finally, possible scar tissue located in the cavity muscle is automatically detected and visualized on the 3D heart model. The model is exported in format which is compatible with interventional devices at hospital. The evaluation of the software tool included two steps. Firstly, a comparison with two free software tools was performed on two image data sets of variable quality. Secondly, six scientists and physicians tested our tool and filled out a questionnaire. The performance of our software tool was visually judged more satisfactory than the free software, especially on the data set of lower quality. Professionals evaluated positively our functionalities regarding time taken, ease of use and quality of results. Improvements would consist in performing the planning based on different MR modalities

Comparison of spectral characteristics in human and pig biliary system with hyperspectral imaging (HSI)

Injuries to the biliary tree during surgical, endoscopic or invasive radiological diagnostic or therapeutic procedures involving the pancreas, liver or organs of the upper gastrointestinal tract give rise to the need to develop a method for clear discrimination of biliary anatomy from surrounding tissue. Hyperspectral imaging (HSI) is an emerging optical technique in disease diagnosis and image-guided surgery with inherent advantages of being a non-contact, non-invasive, and non-ionizing technique. HSI can produce quantitative diagnostic information about tissue pathology, morphology, and chemical composition. HSI was applied in human liver transplantation and compared to porcine model operations to assess the capability of discriminating biliary anatomy from surrounding biological tissue. Absorbance spectra measured from bile ducts, gall bladder, and liver show a dependence on tissue composition and bile concentration, with agreement between human and porcine datasets. The bile pigment biliverdin and structural proteins collagen and elastin were identified as contributors to the bile duct and gall bladder absorbance spectra